Skip to content

fix(analyzer): bound native-build registry concurrency - #3988

Merged
JSONbored merged 1 commit into
mainfrom
codex/propose-fix-for-native-build-analyzer-issue
Jul 7, 2026
Merged

fix(analyzer): bound native-build registry concurrency#3988
JSONbored merged 1 commit into
mainfrom
codex/propose-fix-for-native-build-analyzer-issue

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • Prevent the native-build analyzer from launching the entire capped set of registry fetches concurrently, which can cause large bursts of network/parse/memory work when many queryable dependency changes appear in a PR.
  • Preserve the existing total query cap while limiting simultaneous outbound registry requests to a small, safe concurrency level.

Description

  • Add a new concurrency cap MAX_CONCURRENT_REGISTRY_QUERIES = 4 in review-enrichment/src/analyzers/native-build.ts to limit simultaneous registry calls.
  • Introduce an order-preserving bounded-concurrency helper mapWithConcurrency and replace the previous Promise.all(changes.map(...)) fanout with mapWithConcurrency(changes, MAX_CONCURRENT_REGISTRY_QUERIES, ...) inside scanNativeBuild so lookups are throttled.
  • Keep existing caps and byte limits (MAX_QUERIES, MAX_NPM_VERSION_JSON_BYTES, MAX_PYPI_VERSION_JSON_BYTES) unchanged and retain result ordering and behavior of scanNativeBuild and npmNativeBuild.
  • Add a regression unit test in review-enrichment/test/native-build.test.ts that submits multiple PyPI additions and asserts that all are processed while the observed maxActive concurrent fetches never exceeds the concurrency cap.
  • Small TypeScript typing adjustment to avoid T | undefined being passed into the worker (items[index]!) to satisfy the compiler.

Testing

  • Ran npm --prefix review-enrichment run build (REES build) — succeeded.
  • Ran the native-build unit file with node --test --experimental-strip-types review-enrichment/test/native-build.test.ts — all tests in that file passed, including the new concurrency regression test.
  • Ran the REES test suite via npm run rees:test — review-enrichment tests passed locally.
  • Started the full local gate with npm run test:ci; the REES pieces passed but the full test:ci run failed in unrelated root-level coverage/unit shards (root test:coverage encountered timeouts / failures in test/unit/queue.test.ts unrelated to this change).
  • npm audit --audit-level=moderate could not complete in this environment due to the registry returning 403 Forbidden for the audit endpoint.
  • The change is minimal and focused to bound concurrency without changing analyzer semantics; the added unit test exercises the new behavior and passed.

Codex Task

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored
JSONbored merged commit 0f5e980 into main Jul 7, 2026
8 checks passed
@JSONbored
JSONbored deleted the codex/propose-fix-for-native-build-analyzer-issue branch July 7, 2026 09:15
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant